Software architects often engage in a process of improving software after deployment of the software. The improvements may be implemented by modifying a version of the software or by creating a new version of the software, where the modified or new version of the software is intended to replace the deployed (current) version of the software. Deployment of the modified or the new version of the software may have an impact on hardware that supports the version of the software (e.g., require more or less processing power and/or time), may impact outcomes resulting from user interaction (e.g., satisfy, annoy, or frustrate users, etc.), or may have other possible outcomes (e.g., include bugs, etc.). Therefore, it is desirable to perform a comparison test, often called A/B testing, to compare results following execution of the modified or new version of the software against results following execution of the deployed (current) version of the software prior to a full deployment of the modified or new version of the software.